Planning Medical Therapy Using Partially Observable Markov Decision Processes
نویسندگان
چکیده
Diagnosis of a disease and its treatment are not separate, oneshot activities. Instead they are very often dependent and interleaved over time, mostly due to uncertainty about the underlying disease, uncertainty associated with the response of a patient to the treatment and varying cost of different treatment and diagnostic (investigative) procedures. The framework particularly suitable for modeling such a complex therapy decision process is Partially observable Markov decision process (POMDP). Unfortunately the problem of finding the optimal therapy within the standard POMDP framework is also computationally very costly. In this paper we investigate various structural extensions of the standard POMDP framework and approximation methods which allow us to simplify model construction process for larger therapy problems and to solve them faster. A therapy problem we target specifically is the management of patients with ischemic heart disease.
منابع مشابه
On the Undecidability of Probabilistic Planning and Infinite-Horizon Partially Observable Markov Decision Problems
We investigate the computability of problems in probabilistic planning and partially observable infinite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic finite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there ex...
متن کاملOn the Undecidability of Probabilistic Planning and Innnite-horizon Partially Observable Markov Decision Problems
We investigate the computability of problems in probabilistic planning and partially observable innnite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic nite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exist...
متن کاملOn the Undecidability of Probabilistic Planning and In nite-Horizon Partially Observable Markov Decision Problems
We investigate the computability of problems in probabilistic planning and partially observable innnite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic nite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exist...
متن کاملPlanning with Partially Observable Markov Decision Processes: Advances in Exact Solution Method
There is much interest in using par tially observable Markov decision processes (POMDPs) as a formal model for planning in stochastic domains. This paper is concerned with finding optimal policies for POMDPs. We propose several improvements to incre mental pruning, presently the most efficient exact algorithm for solving POMDPs.
متن کاملPartially Observable Markov Decision Processes with Continuous Ob- servations for Dialogue Management
This work shows how a dialogue model can be represented as a Partially Observable Markov Decision Process (POMDP) with observations composed of a discrete and continuous component. The continuous component enables the model to directly incorporate a confidence score for automated planning. Using a testbed simulated dialogue management problem, we show how recent optimization techniques are able...
متن کامل